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Executive Summary 



This report provides a summary of six summative evaluation studies that were 
implemented and completed between 1999 and 2002. The evaluations were conducted 
on three different streams of Canada’s Youth Employment Strategy (YES). 

The Youth Employment Strategy was introduced by the federal government in 1997 to 
address employment related challenges facing youth. Fourteen federal departments and 
agencies together with public, private and voluntary sector partners were involved in the 
delivery of YES during the 1997 to 2002 period. 

The main goal of the Strategy is to assist young Canadians to prepare for, obtain, 
and maintain employment and to make a successful transition from school to work. 
YES focuses on young Canadians aged 15-30 who are unemployed or underemployed. 

During the 1997-2002 period, there were approximately 35 programs arranged in 
five major streams in YES. Each of the six evaluation studies that are summarized in this 
report addresses one or several aspects of these streams. 



Background 

The Youth Employment Strategy Streams 

During the 1997 to 2002 period, YES consisted of five major streams. 

• Science and Technology Internships focused primarily on post-secondary graduates 
who were unemployed or underemployed. Its overall aim was to provide these young 
Canadians with a first career-related work experience in key job-generating sectors. 

• Youth International Internships also focused primarily on post-secondary graduates 
who were unemployed or underemployed. YI internships were designed to provide 
youth with internationally focused work experience that would assist them in making a 
successful school-to-work transition. 

• First Nations and Inuit Youth Internships provided youth on reserve and in 
recognised communities with work-related skills and experience to help them make the 
transition from school to work. These programs focused on helping youth develop 
skills, obtain work experience and acquire entrepreneurial abilities that reflect job 
prospects on reserves or in local labour markets. 

• Student Summer Placements helped youth who were returning to school to obtain 
career-related summer work experience and to earn income to further their education. 
Employers from the private, public and not-for-profit sectors received a wage subsidy 
for hiring students for a minimum of 6 to a maximum of 16 consecutive weeks. 
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• Other Government of Canada initiatives for youth included, among others, 
HRDC’s Youth Internship Canada and Youth Service Canada. These two initiatives 
existed prior to 1997 and were integrated under the YES. 

- Youth Internship Canada (YIC) was a community -based initiative designed to 
create entry-level opportunities for young people to assist them in making an 
effective transition from school to work. YIC was intended to enable out-of-school, 
unemployed or underemployed young people to gain employability skills and 
practical work experience to enhance their ability to get a job after the internship. 

- Youth Service Canada was designed to provide unemployed and out-of-school 
youths between 18 and 24 years of age with an opportunity to gain relevant 
work-related experience while working in teams, typically within a project lasting 
six to nine months. In addition, YSC was designed to provide youths with the 
opportunity to leam or enhance transferable job-related skills; develop life skills 
such as self-esteem, self-reliance, leadership, communication, and team-work; 
and contribute to their community. After integration under the YES, YSC has 
focussed primarily on youth at risk aged between 15 and 30. 

Of the six evaluation studies that are summarized in this report, one addresses the 
Science and Technology stream and another addresses the Youth International stream. 
The remaining four studies are devoted to Youth Service Canada (YSC) and Youth 
Internship Canada (YIC) — two programs delivered by Human Resources 
Development Canada (HRDC) as part of the last stream (i.e., other Government of 
Canada initiatives for youth). 



Overview of Labour Market Conditions and Lessons 
Learned Regarding Employment Programs for Youth 

There were approximately six million Canadians between 15 and 29 years old in 2001. 
Of these, four million were between 15 and 24; about two million were between 15 and 19. 

Transitions into and within the labour market can be difficult for any worker. A number 
of factors that make these transitions particularly challenging for youth include 
continuously changing and evolving skill requirements, lack of job experience, the costs 
of skills development, and competition for jobs. For some youth, these challenges are 
compounded by personal issues, making their transitions with respect to the labour 
market even more arduous. 

With respect to the labour market: 

• According to an indicator developed by the OECD, the duration of the school-to-work 
transition in Canada increased between 1985 and 1998. The “average duration” of this 
transition increased from six years (ages 16 to 21 inclusive) in 1985 to eight years 
(ages 16 to 23 inclusive) in 1998; 
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• Between 1989 and 1997, full-time school enrolment increased (from 48.0% to 57.9%) 
and the labour force participation rate fell (from 71.0% to 61.5%) among those aged 
15-24. From 1998 to 2001, the full-time school enrolment rate declined slightly from 
its 1997 level, while the labour force participation rate for these youth rose somewhat. 
In 2001, the full-time school enrolment rate was 56.7% and the labour force 
participation rate was 64.7% for this group; 

• Overall, the unemployment rate for youth was relatively high, compared with those 
between 30 and 64. Between 1997 and 2001, the annual unemployment rate for those 
between 30 and 64 years of age did not exceed 7.5%. It was 5.9% in 2001. The rate for 
those between 15-19 was 21.2% in 1997 and 16.6% in 2001. For those aged 20 to 24, 
it was 13.1% in 1997 and 10.3% in 2001. The rate for those aged 25 to 29 was 9.6% in 
1997 and 7.5% in 2001; 

• The unemployment rate for youth varied with educational attainment. During the 
1990-2001 period, those among youth aged 15-24 who had not completed high school 
had the highest annual unemployment rates, while those who had a post-secondary 
certificate or diploma had the lowest. The annual unemployment rates for youth aged 
15-24 with some high school were 24.0% in 1997 and 18.6% in 2001. By comparison, 
the annual unemployment rates for youth with a post- secondary certificate or diploma 
were 10.9% in 1997 and 7.9% in 2001; 

• The rate of involuntary part-time work among youth aged 15-24 rose from 5% in 1989 
to 10% in 1996. Among post-secondary graduates, 35% to 41% reported that they 
were overqualified for their job. 



Lessons Learned in Employment Programming 
for Youth 

In 1997, Human Resources Development Canada published a Lessons Learned study that 
focused on “what works” in youth employment programming (HRDC, 1997). The study 
was based on a review of available evaluation reports on employment programs and 
interventions delivered to youth in Canada, the United States, Australia and the United 
Kingdom. The review was conducted for HRDC under a contract with experts in 
evaluation methodology and in employment programming (see Social Research and 
Demonstration Corporation, 1996). 

In the Canadian case, some of the findings are that: 

[...] The estimated impacts of these programs on earnings and employability 
of participants are often positive. These generally positive impacts appear to 
be mainly short term [...]; nonetheless, positive (but typically much smaller) 
effects persist over longer periods for some trainees. 

Estimated training impacts are generally largest for those facing the best 
labour market opportunities and smallest for those who face the greatest 
obstacles to obtaining and retaining employment. In addition, the limited 
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available evidence suggests that those facing the greatest labour market 
difficulties are those who are likely to obtain benefits which persist beyond 
a year or two of the training. (Riddell 1996: 50). 

The review of evaluations of youth programs in Australia and the United Kingdom 

concluded that: 

Several evaluations [...] show that participation by young people in 
labour market programs does have a positive net impact. [...] [Sjome 
interx’entions have demonstrated that they can increase the chance of 
getting a job by 50 percent; and, even after six to eight months, 
participants can be more likely than the comparison group to be still 
employed [...]. (Burns and Thomas, 1996: 87). 

Looking at all of the evidence, the Lessons Learned study (HRDC, 1997: 4-6) draws 

general lessons, including the following : 

• In general, labour market success among youth is correlated with educational level; 

• When programs produce positive results, their impacts are generally modest in size. 
The fact that a program has only a small impact does not necessarily mean that it is not 
worthwhile. It may still be cost-effective - producing a positive return on the 
investment of public funds - and may be critical to turning around the lives of 
particular young people; 

• Because young people’s needs are diverse, no single intervention can be expected to 
deal with the full spectrum of needs; 

• Most effective programs for young people provide sustained adult contact. The roles 
played by adults vary from program to program - teacher, mentor, case manager, 
counsellor, or supervisor. The key factors are that there be ongoing contact with an 
adult over an extended period of time and that it includes elements of monitoring, 
as well as support; 

• The most effective strategy for disadvantaged youth combines a training component 
with links to employers, on-the-job training and work experience, and, for the most 
disadvantaged youth, job search assistance and transitional wage subsidies; 

• Interventions targeting high school leavers are most effective when initiated as soon as 
possible after they leave school; 

• Work experience programs have produced generally disappointing long-term results. 
Job creation initiatives that have been successful generally provide meaningful 
employment in sectors of the economy where job growth is expected; 

• The success of youth employment programs generally depends not only on good program 
design and delivery, but also on the presence of job opportunities in the labour market. 
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Evaluation of YES Program Components 



Overview of Methodology 

All six studies relied on designs that involved comparison groups. In these designs, 
the outcomes of groups of participants are compared with those of non-participants 
comparison groups. Non-participants are selected separately from participants on the 
basis of their relative similarity to the latter. 

In each of the YES evaluation studies, information on the two groups was collected 
through telephone surveys. In addition to socio-demographic characteristics (age, gender, 
education, etc.), the surveys were used to collect information on participants’ perceptions 
of the programs and on their labour market activities and earnings in the pre- and 
post-program periods. In particular, these included 1) participant satisfaction with key 
elements of the program design; and 2) perceptions of program participants regarding 
the effects of participation on selected decisions they made and employment outcomes 
they experienced in the post program period, in particular job and career choices and 
skills development. 

Moreover, using econometric techniques, the evaluations estimated impacts of the 
program with respect to employment, earnings, time in school, and reliance on 
Employment Insurance (El). Regression techniques were used to estimate program 
impacts. These techniques were applied because of their ability, under certain conditions, 
to identify the relationship between program participation and the outcome of interest 
(for instance: earnings, employment, etc.) while taking into account other observed 
influential characteristics (e.g., age, gender, etc.). 



Summary of Evaluation Results 

Participant Satisfaction with the Programs 

Participant satisfaction was examined in some form in each of the four programs. 

A majority of program participants in S&T, YI and YIC (Phase III) who responded to 
the surveys stated that they were satisfied with the information they had received about the 
program and the application process. Between 73% and 89% of respondents were satisfied 
with these elements of the program. 

A majority of participants in these three programs were also satisfied with the job matching 
process, the opportunity to learn, and the length of the internship. The percentage of 
respondents who stated that they were satisfied with these elements of the program ranged 
between 83% and 94%. A majority of respondents (between 69% and 78%) were satisfied 
with the rate of pay they received during their internship. 
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V 




For YSC, 92% of the participants who responded to the survey said that they were 
“fairly” or “very” satisfied with their internship. 

Program completion rates ranged from 79% to 88%. 

As discussed in the technical reports of the individuals studies, according to other sources 
of information (focus groups and interviews), sponsors, employers and partners were also 
satisfied with these elements of the program. 

Participants Perceptions and Selected Outcomes 

All six evaluations examined participant perceptions using information collected in 
surveys. Respondents to the S&T, YI and YIC Phase III surveys were asked about their 
general level of employability. They were asked to rate their employability prior to 
participating in the program, as well as their current level of employability at the time of 
the summative evaluation survey. According to the post-program survey, between 23% 
and 31% of the respondents rated their pre-program employability level as “high” or 
“very high”. The corresponding post-program percentages ranged between 81% and 85%. 

Survey respondents in four of the studies were asked whether or not they felt that they 
enhanced or developed specific work-related skills during their involvement with the 
program. Generally, respondents indicated that they had. With respect to adaptability, 
over 90% stated that they had improved this skill. About 90% of respondents felt that 
they were more self-motivated than before the program, and between 86% and 93% felt 
that their teamwork skills had improved. 

For many, computer and numerical skills also improved. According to survey 
respondents’ self-assessments, the proportions who perceived it that way varied among 
programs. A much higher percentage (90%) of S&T participants stated that their 
computer skills had improved during their internship program, than participants (between 
55% and 67%) in other programs. A higher percentage of respondents to the S&T and 
YIC Phase III surveys indicated that their numerical skills had improved during their 
program (65% and 58%, respectively), than those in YI (41%). This appears to be in line 
with the objectives of the programs and the needs of the clients. 

Between 46% and 80% of S&T, YI and YIC survey respondents felt that their skills in 
mathematics and science and their technical research skills had improved during their 
participation in the internship program. 

Between 30% and 49% of respondents to the S&T, YI, and YIC Phase I and Phase III 
surveys stated that they returned to school after their internship. Among these 
respondents, 49% to 64% stated that the internship influenced this decision. 

A majority of S&T, YI, and YIC Phase III survey respondents (between 78% and 85%) 
indicated that the internship helped “somewhat” to “a lot” to determine their career choice. 
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Respondents were also asked whether they thought they were underemployed in terms of 
existing and most recent job for two periods: at program entry and at the time of the 
post-program survey. According to their self-assessments, the percentage of respondents 
who stated that they were underemployed dropped over time. The percentage of 
respondents who stated that they were underemployed prior to or at entry to the program 
ranged from 49% to 68%; the percentage reporting underemployment at the time of the 
post-program survey ranged from 24% to 30%. 

A majority of respondents generally indicated that the internships were useful for getting 
their most recent job (between 72% and 86%). However, a majority of them (between 
65% and 77%) also believed that they would have found a job even if they had not 
participated in the programs. 

The proportion of participants who said they were unemployed and looking for work at 
the time of the post-program survey was consistent across components. It ranged 
between 10% and 13%. 

By design, the YES components provided participants with direct contacts with employers 
or sponsors. In many cases (between 48% and 56% of respondents), these contacts 
involved some form of mentorship and outlasted the internships. A majority 
of participants continued to have contacts with their program employers after the end of 
their internships (between 52% and 68%). However, many stated that they had already 
had contact with their internship employer prior to entering the program (the highest was 
52%, in the case of S&T). Moreover, certain participants had already worked for the 
employer prior to entering the program (the highest proportion was 25%, for S&T). 

Program Impacts on Employment Outcomes: Results of 
Econometric Analyses 

As stated earlier, each of the studies included the analysis of data collected in the 
surveys and the use of econometric techniques to address the question of program 
impacts. The econometric techniques were applied to survey data in order to compare 
post-program employment-related outcomes of participants with those of non-participants. 
The econometric analyses of respondents to the surveys suggest the following: 

• Overall, earnings were estimated to be higher for participants in S&T, YI and YIC than 
for corresponding non-participants. It was estimated that annualized earnings 
differences ranged from $2,299 to $5,724, depending on the program. The YSC study 
(a year 2000 follow-up of 1995-1997 participants) did not detect a difference in 
earnings between the two groups. 

• Three of the four studies provided estimates of pre-to-post gains in earnings by males 
and females. The S&T study suggests that female participants gained by $9,420 on 
average. The YI and YIC studies suggest that female participants gained $4,854 and 
$3,400 respectively. According to the estimates, the S&T, YI and YIC studies did not 
detect earnings gains for men. The YSC study did not detect a difference in earnings 
for women or for men. 
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• Regarding the percent of time employed, the studies provide mixed results. 
Participants in YI and YIC are estimated to have spent more time in full-time 
employment than non-participants (8.5 and 15.9 percentage points, respectively). 
According to the estimates, there was no detectable difference between participants in 
S&T and their non-participant counterparts. Similarly, there was no detectable 
difference for the YSC participants and non-participants. While the analysis suggests 
that S&T participants did not spend more time in full-time employment than their 
non-participant counterparts, participants in that program did appear to spend more 
time in school than non-participants (5.7 percentage points more time per year, 
on average, on the part of participants). 

• According to the estimates, with the exception of YIC, the evaluations did not detect 
any difference in the number of weeks on El between program participants and 
non-participants. 



Overall Assessment and Conclusion 

The evaluations’ results are mixed. Generally, a majority of participants who responded 
to the surveys expressed a high level of satisfaction with key program elements. 
In addition, a majority of participants stated that participation in the program increased 
their employability and improved skills. Participants were also less likely to state that 
they were underemployed. However, many respondents did state that they would have 
found jobs even without the program, and many indicated that they had already had 
contact with their internship employer prior to the program. The econometric analysis on 
earnings, employment and unemployment suggests that programs had moderate 
employment-related gains for some youth. These tended to be the youth who participated 
in the programs that involved the more highly educated youth - the ones who participated 
in the Science and Technology, Youth International, and Youth Internship Canada 
streams. There were little to no gains for the group that participated in YSC before it was 
redesigned with greater focus on youth at risk. Overall, these results could support a 
reconsideration of selected aspects of the programs. 



VIII 
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Management Response 



Canada’s Youth Employment Strategy (YES) announced by the Minister of Human 
Resources Development Canada in February 1997 marked a new way for the government 
to provide young people with the necessary skills and work experience they need to be 
better prepared to make a transition into the labour market. 

As of April 2003, the YES has been realigned. It has moved from a set of some 35 discrete 
initiatives to three integrated programs offered by 14 departments. The programs are: Skills 
Link, Career Focus and Summer Work Experience. 

This report provides a summary of findings of six evaluation studies conducted between 
1997 and 2002 of previous YES programs. The studies examined approximately 
35 programs offered by 14 participating departments through the YES. The individual 
studies provided an opportunity for participating federal departments to look at the 
delivery and impact of employment initiatives provided to youth under YES. 

Overall, the evaluation findings indicated youth and employers were satisfied with their 
involvement in the program and that many youth had enhanced their employability skills 
by program completion. The majority of participants were satisfied with the job matching 
process, the opportunity to learn, and the length of the internship, and a majority of 
respondents were satisfied with the rate of pay they received during their internship. 
In addition, there was a moderate positive impact for youth with some employability 
skills. However, this was not the case for youth with lower skills. 

The findings did imply areas for improvement, particularly with regards to incrementality 
and employment gains. With regards to incrementality, many respondents stated that they 
would have found jobs even without the program and that they had already had contact 
with their internship employer prior to the program. The findings were based on 
participant survey. 

The shift in program design away from the “project based” approach to one that focuses 
on meeting the needs of individuals is being facilitated through client assessment and 
case management. Client assessment and case management clearly identifies the 
individual needs, skills gaps and goals and is the basis for the employment action plan. 
This approach to tailored program interventions for individuals is designed to address 
the issues of program incrementality and would facilitate ultimate full participation in the 
labour market. 

Secondly, the econometric analysis on earnings, employment and unemployment 
suggests that programs had only modest employment-related gains for some youth. 
The shift to a client-centred program design will respond to this evaluation finding, and is 
supported by the findings of additional research and surveys. In addition to employment, 
the new YES programs design focuses on the value of, and the need for, skills necessary 
to compete in the labour market of the 21st century, including the acquisition of further 
education. Also, the program design aims at employment related gains and time spent in 
the labour market over a longer period of time as opposed to a shorter period time. 
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Under Skills Link, youth facing barriers to employment are being provided with longer 
term interventions including employability, life and occupational skills acquisition and 
work experiences leading to full labour market participation thus employment-related 
gains. For those youth requiring substantial assistance and multiple interventions, case 
management will ensure that individual employment action plans are on target and are 
adjusted as required. 

The Career Focus program provides post secondary graduates with tailored, career- 
relevant work experience that maximizes their academic skills and supports the 
advancement of practical skills that will enable them to become leaders, or encourage 
them to pursue further post-secondary education. 

Both Skills Link and Career Focus will identify skills gaps and address the needs of 
industry to ensure the skills of youth clients are maximized to their full potential. Both 
components are being used to ensure a better match between youth and the career-related 
work experience opportunities provided by sponsors. 

These new features are designed to address the concern of the evaluation. 



x 
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1. Introduction 



This report provides a summary of six summative evaluation studies that were 
implemented and completed between 1999 and 2002. The evaluations were conducted on 
three different streams of Canada’s Youth Employment Strategy (YES). 

The Youth Employment Strategy was introduced by the federal government in 1997 to 
address employment related challenges facing youth. Fourteen federal departments and 
agencies together with public, private and voluntary sector partners were involved in the 
delivery of YES during the 1997 to 2002 period. 

The main goal of the Strategy was to assist young Canadians to prepare for, obtain, 
and maintain employment and to make a successful transition from school to work. 
YES focused on young Canadians aged 15-30 who were unemployed or underemployed. 

This report provides a description of each of the YES streams (Chapter 2), a background 
with respect to labour market conditions and lessons learned elsewhere (Chapter 3), 
an outline of the evaluation methodology, basic profile of YES participants who were 
surveyed as part of the evaluation studies, and a summary of each of the studies results 
(Chapter 4), and an overall assessment and conclusions (Chapter 5). 
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2. The Youth Employment Strategy 



During consultations with Canadians across Canada during the spring of 1996, a Ministerial 
Task Force on Youth heard young people, as well as employers and other Canadians, say that 
the biggest challenge youth face is to get a first job without prior work experience. 
In response to this and other employment-related challenges faced by Canadian youth, 
the Government of Canada launched its Youth Employment Strategy (YES) in 1997. 

The main goal of the Strategy was to assist young Canadians to prepare for, obtain, and 
maintain employment and to make a successful transition from school to work. 
Specifically, the YES was designed to help young Canadians get the information, 
knowledge, skills and work experience they need to make career choices, pursue learning 
and prepare for and participate in the world of work. YES focused on young Canadians 
aged 15-30 who were unemployed or underemployed. 

The YES was designed to build on and integrate existing federal government programs 
targeted at youth. The delivery of the YES during the 1997 to 2002 period involved a 
partnership of fourteen federal departments and agencies, and was primarily administered 
through contribution agreements between these departments and private, non-profit and 
municipal sectors. There were approximately 35 programs arranged in five major streams: 

• Science and Technology Internships (S&T); 

• Youth International Internships (YI); 

• First Nations and Inuit Youth Internships; 

• Student Summer Placements; and 

• Other Government of Canada Initiatives for Youth. 

In all, the six evaluation studies that are summarised in this report address the first 
two streams and HRDC’s components of the fifth stream. Specifically, one addresses 
the Science and Technology stream and one addresses the Youth International stream. 
The remaining four are devoted to Youth Service Canada (YSC) and Youth Internship 
Canada (YIC) — two programs delivered by Human Resources Development Canada 
(HRDC) as part of the last stream: Other Government of Canada Initiatives for Youth. 1 

The remainder of this chapter provides additional information on each of the streams. 



2.1 Science & Technology 

Nine departments and agencies delivered internship programs within the framework of 
the Science and Technology (S&T) stream during the 1997 and 2002 period. This stream 
primarily focused on post-secondary graduates who were unemployed or underemployed. 



i 



The technical evaluation reports upon which this summary is based are referenced at the end of this report. 
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Its overall aim was to provide these young Canadians with a first career-related work 
experience in key job-generating sectors. 

The Science & Technology stream included the Technology stream, the Science Horizon 
stream, and the National Sector Council stream. The Technology stream was to provide 
interns with experience in the information technology sector by working in small and 
medium-sized enterprises that focus on the development and/or application of 
technology. The Science Horizons stream was to provide young scientists with work 
experience. The National Sector Council stream was delivered by Human Resources 
Development Canada via national sector councils. 

The federal departments worked with both sponsors and employers to ensure that interns 
obtained suitable science- and technology-related work. 

During fiscal years 1998-99 and 1999-2000, a total of 6,890 youth participated in the 
S&T stream. Approximately $20 million was allocated to the S&T program in each of 
these two years. 



2.2 Youth International 

Youth International (YI) also focused on post- secondary graduates who were unemployed 
or underemployed. YI internships were designed to provide youth with internationally- 
focused work experience to assist them in making a successful school-to-work transition. 
The average duration of these internships was approximately seven and a half months. 

The YI component was delivered by six federal departments and agencies. Departments 
worked with sponsoring organisations to provide work experience. Funding from the 
federal government was intended as a contribution toward interns’ salaries and/or various 
administrative and travel costs associated with the internship. 

A total of 4,763 youth participated in YI internships during the fiscal years 1998-1999 
and 1999-2000. Approximately $30 million was allocated to YI in each of these 
two years. 



2.3 First Nations and Inuit Youth Internships 

These internships provided First Nations and Inuit youth on reserve and in recognised 
communities with work-related skills and experience to help them make the transition 
from school to work. These programs focused on helping youth develop skills, obtain 
work experience and acquire entrepreneurial skills that reflected job prospects on 
reserves or in local labour markets. For example, Indian and Northern Affairs operated 
the First Nations and Inuit Youth Work Experience Program which funded projects 
proposed by First Nations and Inuit organisations. The purpose of these projects was to 
provide supervised work experience to out-of-school unemployed youth in community 
service, community businesses or other work experience. The First Nation and Inuit 
Internships served youth who were between 16 and 24 years of age and who lived on 
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reserve or in recognised communities. Youth were generally provided with mentored 
work opportunities in order to increase their prospects for long term employment and to 
strengthen their communities. 

In fiscal years 1997-98 and 1998-99, there were 13,236 participants in the First Nations 
and Inuit stream. Expenditures for these two fiscal years were $30 million. 



2.4 Student Summer Placements 

Student Summer Placements helped youth returning to school to obtain career-related 
summer work experience and to earn income to further their education. Employers from 
the private, public and not-for-profit sectors received a wage subsidy for hiring high 
school, college or university students for a minimum of 6, to a maximum of 16 weeks. 
The work experience was combined with supervision, mentoring. 

This summer employment program had 148,830 participants in the two fiscal years 
1997-98 and 1998-99, and expenditures were approximately $240 mi llion for these 
two years. 

Summer Career Placement, an HRDC program under this stream, was the subject of an 
evaluation study published in 1997 (see HRDC, 1997b). 



2.5 HRDC’s Youth Initiatives under YES 



2.5. 1 Youth Internship Canada 

Youth Internship Canada (YIC) was a community-based initiative designed to create 
entry-level opportunities for young people to assist them in making an effective transition 
from school to work. YIC was intended to enable out-of- school, unemployed or 
underemployed young people to gain employability skills and practical work experience 
to enhance their ability to get a job after the internship. Federal funds were provided by 
HRDC to sponsoring organisations to develop projects that offered youth internships 
in their local labour market. YIC internships were normally five to seven months in 
duration. YIC existed as an HRDC initiative prior to the announcement of the YES, 
and was integrated as part of the Strategy in 1997. 

During 1998-99 and 1999-2000, over 53,000 youth participated in YIC. Expenditures 
over these combined fiscal years were approximately $166.4 million. 
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2.5.2 Youth Service Canada 



Youth Service Canada (YSC) is the official name given to the youth service corps that 
had been announced in the Speech from the Throne in January 1994. YSC was designed 
to provide unemployed and out-of-school youth between 18 and 24 years of age with an 
opportunity to gain relevant work-related experience while working in teams (typically 
within six-to-nine month projects). 

YSC became part of the YES in 1997 and has focussed, since then, primarily on youth 
at risk aged between 15 and 30. Projects were usually co-ordinated and managed by 
not-for-profit sponsor organisations that had developed the proposals that were approved 
by Human Resources Development Canada. 

In addition to real work experience, YSC was designed to provide young Canadians with 
the opportunity to: 

• learn or enhance transferable job-related skills; 

• develop personal qualities and skills such as self-esteem, self-reliance, leadership, 
communication, and team-work; and 

• contribute to their community and country. 

In the fiscal years 1997-98 and 1998-99, there were 11,899 participants. Expenditures for 
these two years combined were approximately $100 mi llion. 

Unlike the other studies, the YSC evaluation focused on youth who entered the program 
during an earlier period, between June 1995 and March 1997 (i.e. before YSC was 
redesigned and integrated as part of the YES). See HRDC (1999) for an evaluation of 
short-term outcomes of the same cohort. 
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3. Labour Market Profile and 
Lessons Learned Regarding Employment 

Programs for Youth 



There were approximately six million Canadians between 15 and 29 years of age in 2001; 
of these, four million were between 15 and 24, about two million were between 15 and 
19. Youth aged 15-24 — and to some extent, those aged 25-29 — may be considered as 
undergoing a number of transitions, one of which is the school to work transition. At the 
individual level, according to Thiessen (2001), “The nature, timing, and sequence of 
previous transitions constrain or facilitate later positive outcomes.” It is generally 
accepted that the transition between school and work is an important one in determining 
future outcomes. 

Transitions into and within the labour market can be difficult for any worker. A number 
of factors that make these transitions particularly challenging for youth include ever 
changing and evolving skill requirements, lack of job experience, the costs of skills 
development, and competition for jobs. For some youth, these challenges are compounded 
by personal issues, making their transitions with respect to the labour market even 
more arduous. 

This chapter provides a labour market profile of youth over the past 20 years 
(Section 3.1) and conclusions from a lessons learned study of youth programming in 
Canada and abroad (Section 3.2). 



3.1 Youth and the Labour Market 

3.1.1 The School -to- Work 
Transition - Average Duration 

The Organisation for Economic Co-operation and Development (OECD) has formulated 
an indicator for measuring the “average duration” of the school-to-work transition. 
According to this indicator, the school-to-work transition can be defined as the process 
that takes place between (a) the lowest age at which youth are no longer predominantly 
(i.e., less than 75%) studying and not working; and (b) the lowest age at which the 
majority of them (i.e., more than 50%) are working but not studying (OECD, 1997; 
Bowlby, 2000). In Canada, the span between these two ages increased between 1985 and 
1998 (Figure 3-1). In 1985, the estimated duration of the school-to-work transition 
process was six years, beginning at the age of 16 and ending at the age of 21. 
The comparable estimate for 1998 is eight years, beginning at the age of 16 and ending at 
the age of 23. 
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Figure 3-1 

The School-to-Work Transition*, Canada, 1984-1998 




• Start of transition ■ End of transition 



* Based on eight-month averages, excluding summer months. 

Source: Statistics Canada, Labour Force Update, Autumn 1999, catalogue no. 71-005-XPB. 



3. 1.2 Labour Force Participation and School Enrolment 

During most of the eighties, there was a steady increase in the labour force participation 
and the full-time school enrolment rates of youth aged 15-24 (Figure 3-2). Between 1989 
and 1997, full-time school enrolment continued to increase (from 48.0% to 57.9%). 
However, the labour force participation rate fell (from 71.0% to 61.5%) during the same 
period. In part, this decline may be attributed to choices made by youth to stay in school 
longer and to cyclically weak employment (Archambault and Grignon, 1999; HRDC, 
2000b). From 1998 to 2001, the full-time school enrolment rate declined slightly from its 
1997 level, while the labour force participation rate of this group rose somewhat. 
In 2001, the participation rate was 64.7% and the full-time school enrolment rate was 
56.7% for this group. 
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Figure 3-2 

Labour Force Participation Rate and Full-time Student Enrolment Rate 
for Youth Aged 15-24, Canada, 1980-2001 




* Annual average. ** Eight-month average, excluding May to August. 

Source: Statistics Canada, Labour Force Historical Review, 2001 version, catalogue no. 71F0004XCB, tables 
CD1T01 AN and CD1T03MN. 



3.1.3 Unemployment 

During the 1980 to 2001 period, youth aged 15 to 29 as a group experienced annual rates 
of unemployment that were consistently higher than for those who were between 30 and 
64 years of age (see Figure 3-3). 

Between 1997 and 2001, the annual unemployment rate for those between 30 and 64 did 
not exceed 7.5%. It was 5.9% in 2001. Over the same period, the annual rate for youth 
between the ages of 15 and 19 dropped from 21.2% to 16.6%. The annual rate for those 
aged 20 to 24 dropped from 13.1% to 10.3%, and there was a drop from 9.6% to 7.5% for 
those aged 25 to 29. 

The unemployment rate for youth varied with educational attainment (Figure 3-4). 
During the 1990-2001 period, those among youth aged 15-24 who had not completed high 
school had the highest annual unemployment rates, while those who had a post- secondary 
certificate or diploma had the lowest. The annual unemployment rates for youth aged 
15-24 with some high school were 24.0% in 1997 and 18.6% in 2001. By comparison, 
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the annual unemployment rates for youth with a post-secondary certificate or diploma were 
10.9% in 1997 and 7.9% in 2001. 



Figure 3-3 

Unemployment Rate* by Age Group, Canada, 1980-2001 




* Annual average. 

Source: Statistics Canada, Labour Force Historical Review, 2001 version, catalogue no. 71F0004XCB, table 
CD1T01 AN. 
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Figure 3-4 

Unemployment Rate* by Level of Education for Youth Aged 15-24, Canada, 1990-2001 




* Annual average. 

Source: Statistics Canada, Labour Force Historical Review, 2001 version, catalogue no. 71F0004XCB, table 
CD1T02AN. 



3. 1.4 Underemployment among Youth 

Two measures are usually used in the literature when discussing underemployment 
among youth. One measure is the proportion of involuntary part-time workers in the 
labour force. For this measure, involuntary part-time workers are defined as those who 
are willing and able to work full-time, but are unable to find full-time employment 
(this measure does not include, for example, those who are working part-time in order to 
attend school or raise children). The other measure is the proportion of youth who 
express the belief that their s ki lls are being under-utilised in their current job. 

Youth have been over-represented in the ranks of involuntary part-timers (in relation to 
their share of the labour force). The rate of involuntary part-time work among youth 
aged 15-24 rose from 5% in 1989 to 10% in 1996. The involuntary part-time rate for the 
group of 25-34 year olds rose from 3% to 4% over the same period. The rate of 
involuntary part-time work is slightly higher for youth with lower educational attainment. 
The rate for youth with high school completion or less was 10% in 1996, compared to a 
rate of 9% for college graduates and 8% for university graduates (Statistics Canada and 
Council of Ministers of Education, Canada, 2000). 
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Using data from the National Graduate Surveys, Finnie (2001) provided estimates of the 
proportion of employed post- secondary graduates who had a higher level of education 
than was required at the start of their job. These estimates are based on the perception of 
youth regarding their own circumstances. Among post- secondary graduates for all 
educational levels taken together (college, Bachelor’s, Master’s and Ph.D.), 35% to 
41% reported that they were overqualified for their jobs in terms of the required level of 
education. Among 1990 college graduates (the most recent cohort studied at the time), 
69% of male college graduates and 75% of female college graduates reported that they 
were overqualified in 1995. 



3.2 Youth and Employment Programs 

In 1997, Human Resources Development Canada published a Lessons Learned study that 
focused on “what works” in youth employment programming (HRDC, 1997). The study 
was based on a review of available evaluation reports on employment programs and 
interventions delivered to youth in Canada, the United States, Australia and the United 
Kingdom. The review was conducted for HRDC under a contract with experts in 
evaluation methodology and in employment programming (see Social Research and 
Demonstration Corporation, 1996). 

Covering about two decades, the Canadian evaluation studies that were reviewed relied 
on non-experimental methods and varied in their degree of technical sophistication 
(Riddell, 1996). The reviews of programs delivered in the United States, Australia and 
the United Kingdom focused on evaluation studies that used randomly assigned groups to 
programs and control groups at program entry, or program participant groups and 
comparison groups selected after the program had been implemented (Long, 1996; Burns 
and Thomas, 1996). 

The review shows that the evaluations involved a variety of programs and interventions 
and groups of young people with diverse needs. It demonstrates that the estimates of the 
effects of the programs and interventions were not uniform. This lack of uniformity may 
be manifested across the variety of combinations of services and clients as well as the 
evaluation methodologies used to derive those estimates.” 

The following provides a summary of the experts’ observations regarding the evaluations 
they reviewed. With respect to the Canadian evidence on training programs, some of the 
findings are that: 

[...] The estimated impacts of these programs on earnings and 
employability of participants are often positive. These generally positive 
impacts appear to be mainly short term (one or two years following 
training) in nature; nonetheless, positive (but typically much smaller) 
effects persist over longer periods for some trainees. 



2 As one of the reviewers put it: “In light of the evolution of evaluation techniques over the past couple of decades, 
it is perhaps not too surprising that the program evaluation studies reviewed for this report employ a wide range of 
methodologies. However, the variation in the reliability of the findings from such diverse studies increases the 
difficulty of generalizing from the reported results.” (Schwartz, 1996: 4). 
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Estimated training impacts are generally largest for those facing the best 
labour market opportunities and smallest for those who face the greatest 
obstacles to obtaining and retaining employment. In addition, the limited 
available evidence suggests that those facing the greatest labour market 
difficulties are those who are likely to obtain benefits which persist beyond 
a year or two of the training. (Riddell 1996: 50). 

The review of evaluations of youth programs in Australia and the United Kingdom 
concluded that: 

Several evaluations which have used carefully-designed comparison 
groups (and, in a few cases, randomly- generated control groups) show 
that participation by young people in labour market programs does have a 
positive net impact. While it is not possible to compare directly the net 
impacts across programs, some interx’entions have demonstrated that they 
can increase the chance of getting a job by 50 percent; and, even after 
six to eight months, participants can be more likely than the comparison 
group to be still employed [...]. (Burns and Thomas, 1996: 87). 

According to the reviewers, the employment trainings programs that were found to 
produce positive effects in Australia and the United Kingdom frequently have the 
following characteristics: they include support services such as job search assistance and 
counselling; they make a careful use of wage subsidies; they attempt to link training with 
work experience; well managed; and they integrate activities of unions, employers, 
trainers/educators and trainees. 

The review of the research on employment, training, and education programs for 
disadvantaged young in the United States “[...] indicates that a number of different 
programs have been effective for in-school youths, while fewer programs have been 
successful for dropouts” (Long, 1996: 55). Among the features of effective programs are 
the following: they involve sustained contact between adults and the youth served by the 
program; they provide financial incentives to succeed and penalize poor performance; 
and they support achievement. Early intervention is also an important element. 

Looking at all of the evidence, the Lessons Learned study (HRDC, 1997: 4-6) draws 
general lessons, including the following: 

• In general, labour market success among youth is correlated with educational level; 

• When programs produce positive results, their impacts are generally modest in size. 
The fact that a program has only a small impact does not necessarily mean that it is not 
worthwhile. It may still be cost-effective - producing a positive return on the 
investment of public funds - and may be critical to turning around the lives of 
particular young people; 

• Because young people’s needs are diverse, no single intervention can be expected to 
deal with the full spectrum of needs; 
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• Most effective programs for young people provide sustained adult contact. The roles 
played by adults vary from program to program - teacher, mentor, case manager, 
counsellor, or supervisor. The key factors are that there be ongoing contact with an 
adult over an extended period of time and that it includes elements of monitoring, as 
well as support; 

• The most effective strategy for disadvantaged youth combines a training component 
with links to employers, on-the-job training and work experience, and, for the most 
disadvantaged youth, job search assistance and transitional wage subsidies; 

• Interventions targeting high school leavers are most effective when initiated as soon as 
possible after they leave school; 

• Work experience programs have produced generally disappointing long-term results. 
Job creation initiatives that have been successful generally provide meaningful 
employment in sectors of the economy where job growth is expected; 

• The success of youth employment programs generally depends not only on good program 
design and delivery, but also on the presence of job opportunities in the labour market. 
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4. Evaluation of YES 
Program Components 



As stated, six summative evaluation studies focused on three streams of the YES and were 
completed between 1999 and 2002. One study was concerned with the Science 
and Technology stream and another with the Youth International stream. The remaining 
four other studies addressed two HRDC youth initiatives: three of them covered the Youth 
Internship Canada (the largest program), and one covered the Youth Service Canada. 

A set of evaluation questions established by the Interdepartmental Evaluation Committee 
(IEC) for internship programs funded under the federal YES guided the evaluation 
studies. Generally, consistent with the objectives of YES, the evaluations investigated 
the extent to which various YES components assisted youth participants in preparing for, 
obtaining and maintaining employment. These were examined in terms of both return to 
school and transition into the labour force. The twelve evaluation questions were divided 
into three broad issues: relevance, program success/achievement of objectives, and 
cost-effectiveness (see Appendix A). Different evaluation phases addressed different sets 
of questions, but the evaluations taken together were intended to address all core issues. 

The issue of program relevance had been addressed in previous evaluations (see HRDC, 
1999; 2000a). The evaluation studies examined mainly questions that pertained to 
program success and, to a limited extent, cost-effectiveness. The summary discusses 
questions related to the issue of program success but does not address cost-effectiveness. 3 

Chapter 4 is divided into three sections. Section 4.1 provides an overview of the evaluation 
methodology. Section 4.2 provides profiles of the participants prior to entering the program, 
and Section 4.3 provides a summary of key results from all the studies. 



4.1 Overview of the Evaluation Methodology in the 
Context of YES 

All six studies relied on designs that involved comparison groups. In these designs, 
the outcomes of a group of participants are compared with those of non-participants 
comparison groups. The data on participants and non-participants were collected through 
surveys. In addition, some of the studies also surveyed employers and sponsors, and/or 
interviewed program officers and conducted focus groups with employers and sponsors. 

This section discusses important properties and relevant aspects of comparison group 
designs, surveys, data analysis and estimation techniques that are typically used in the 
estimation of impacts of employment program. It also provides more specific information 
regarding the evaluation methodology used in each evaluation study. 



3 Due to the limitations in the data, it is difficult to draw conclusions that would support the level of precision often 
implied in results that pertain to cost-effectiveness. 
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4.1.1 Comparison Group Design 

The studies that form the basis of this summary report on the summative evaluations of 
three YES streams. The purpose of summative evaluations is to provide information 
regarding the impact of programs on participants. To do this, evaluation studies examine 
relevant outcomes for participants (after the application of programs) in relation to 
benchmarks designed to represent the outcomes in the absence of the programs. In the 
six evaluation studies summarised in this report, the benchmarks are derived using groups 
of non-participants (called comparison groups) who were selected separately on the basis 
of their relative similarity to participants. 

As discussed below, the selection of the comparison groups was an involved process. 
The validity of estimates based on comparison groups is, in large measure, a function of 
the relevance and quality of the match between the two groups. Any conclusions based 
on estimates of impacts that rely on comparison groups are contingent on the quality of 
the selection of comparison groups. 

Participant groups. The YES was designed for young persons who met pre-determined 
program eligibility criteria. The participant population for each evaluation study 
consisted of persons who met the criteria and actually participated in the programs. 
In order to constitute the participant groups for the YES evaluations, departments and 
agencies were asked to prepare a complete list of participants. In some cases, not all 
departments and agencies that delivered the programs participated in the evaluations. 
Some of those that did participate in the evaluations could not provide complete lists. 

Comparison groups. In five of the studies, the comparison groups were drawn from an 
HRDC database that contains information on individuals who had had some contact with 
a Human Resources Centre of Canada (HRCC). The comparison groups consisted of 
youth who had not participated in a major employment training program up to and during 
the reference period of the evaluations. In the case of the sixth study, the YSC, 
the comparison group was made up of individuals who had completed a baseline survey 
as part of their application for participation in the program but were not selected. 
Administrative and survey data were used to refine the selection of the comparison 
groups (e.g. on the basis of similarity to participants with respect to age, timeframe for 
job seeking, geographic location, and educational background). 



4. 1.2 Surveys 

For each study, participants and comparison group members were contacted for a 
telephone interview. In addition to information on socio-demographic characteristics 
such as age, gender, marital status and education, the interviews were used to collect 
information on participants’ perceptions of the programs and on their labour market 
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activities and earnings in the pre- and post-program periods and similarly defined periods 
for non-participants. 4 

The YES surveys involved samples that constituted subsets of the underlying populations 
of participants and non-participants. 5 As a rule, sampling plans and surveys are designed to 
yield representative samples of the target populations and consequently, reliable estimates 
of outcomes. In practice, deviations from initial plans may affect representativeness. 
They may also reduce the number of units available for analysis to the extent that the 
reliability of estimates may be affected as well. Refusals to respond to survey questionnaires 
and difficulties in contacting the samples are two potential causes for such deviations. 

In the case of the six evaluation studies summarised in this report, as is the case in other 
studies of this type, the group of survey respondents may not be representative members 
of the target population. Where there is a difference, conclusions can be generalised only 
to the subset of the population that is represented by the respondents. 



4 . 1.3 Data Analysis and Estimation 

Comparison group designs are implemented in order to identify net effects of 
the programs (i.e. excluding effects of factors that are not part of those programs). 
The extent to which this goal is achieved is related to the degree of inherent similarity 
between the participant and non-participant samples that are available for analysis. 
In cases where differences do exist, simple comparisons of average outcomes between 
the groups could yield statistically biased estimates of program effects. It may be 
possible to correct for such biases at the analysis stage. Under certain conditions, 6 
regression techniques may be used in order to control for observed differences between 
participants and non-participants. In the case of the YES evaluations, regression was 
used to estimate relationships between program participation and the outcome of interest 
(for instance: earnings and employment) while taking into account other observed 
influential characteristics (e.g., age, gender, etc.). Accordingly, the estimates of impacts 
in this report are based on regression techniques. 

The estimates presented later in this report (Table 4-4 and Table 4-5), include point estimates 
from the regression models and corresponding confidence intervals. Confidence intervals are 
ranges that provide a measure of the precision of the estimate. They can be thought of as 
containing the true population value with a specified level of “confidence” (usually 95%). 



4 Administrative data contained in the El and tax files are usually considered reliable and accurate sources of 
information on labour market activities and earnings. However such data were not available for most evaluation 
studies. Therefore, although the YIC Phase II study included econometric analyses based on El administrative data, 
for comparability purposes, they are not reported here. 

5 It should be noted that the comparison group sample represents a population only indirectly, through the matching 
process. 

6 The validity of the particular regression model's results is contingent on assumptions such as those related to the 
appropriateness of its specification. 

7 In some studies, analytical techniques such as propensity score matching or Heckman's selection model were used 
as a way to control for possible unobserved differences between the participants and the comparison group. 
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Confidence intervals attempt to quantify the degree of confidence in the reliability of results; 
however, they do not take account of unknown biases. 

Generally, statistical reliability of estimates depends on the sizes of the samples involved. 
It is possible for samples to be too small to yield estimates with acceptable reliability. 
In the case of the YES, where possible, certain groups of interest were singled out for 
additional, more detailed (segmented) analysis. The subgroups usually considered in the 
evaluation reports were defined by gender and educational attainment. Clearly, the size 
of the sample that was available for any given subgroup was smaller than the 
one available for the study of the group as a whole. As such, in the cases examined in 
these studies, the analysis of subgroups was often more limited in its potential to provide 
detailed (and “precise”) estimates. 8 



4 . 1.4 Participant and Comparison Group Selection and 
Survey Process 

Not all departments and agencies that delivered programs under the Science and 
Technology and Youth International streams participated in the evaluations. Those that 
participated were responsible for compiling participant lists. Lists obtained from some of 
the participating departments were not complete. Therefore, the number of participants 
available for the evaluation of these streams was significantly smaller than the number 
who participated (as documented in the annual reports). A large number of participants 
on these lists had insufficient contact information. This reduced the size of the initial 
sample selected for analysis and could have affected the degree to which they represented 
the underlying population. 

For the S&T survey, out of the original list of 2,474 interns who participated in 1998-99 
and 1999-2000, 418 participants completed a telephone interview. For the YI survey, 
755 participants completed a telephone interview out of the original list of 3,400 interns 
who participated in the same period. The same comparison group was used in both 
evaluations. A total of 5,566 potential comparison cases were selected from HRDC’s 
administrative files (the National Employment Services System, or NESS). Of these 
cases, 1,000 completed an interview. The survey of comparison group members started 
two weeks after the initiation of the participant survey. This made it possible to monitor 
the selection of comparison group members on the basis of similarities to the sample 
of participants. 



Reliability depends on the degree of homogeneity as well as sample size. This implies that homogeneous subgroups 
have the potential to yield reliable estimates despite “small'’ sample sizes. However, due to non-response and 
incomplete list, potential biases in the subgroups may not be the same as those that may be present in the sample as a 
whole, and the effects of potential biases on the estimates are difficult and often impossible to ascertain. 
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The Youth Internship Canada Phase I participant sample was drawn from a database 
that was constructed by contacting HRCCs and obtaining information on their YIC clients 
who participated in 1997-98. Overall, information was available for approximately 50% of 
the projects. Interviews were completed with 1,439 participants. For the comparison 
group, 58,600 potential members were extracted from NESS. Based on a statistical model 
of program participation, a subgroup of 4,120 potential comparison cases were then 
matched to participants using the nearest neighbour approach. At the time of the survey, 
screening questions resulted in the rejection of a large number of cases. Interviews were 
completed with 554 comparison cases. 

The Youth Internship Canada Phase II study involved a follow-up survey of the 
Phase I respondents who had consented to be re-interviewed. Of the 1,174 participants 
who had consented, 724 completed the Phase II interview in 2001. In the comparison 
group, 228 of the 410 Phase I respondents completed the follow-up survey. 

The Youth Internship Canada Phase III study examined a more recent cohort of YIC 
interns who participated in the program between April 1998 and summer 2000. From a 
sampling frame of 26,500 participants, 10,841 were randomly selected for the initial 
sample. Of these, 2,051 completed an interview. For the comparison group, a sample of 
11,200 potential cases was drawn from NESS and then matched to participants (using a 
statistical model for program participation and the nearest neighbour approach). 
The number of completed interviews for the comparison group was 2,052. During the 
survey, quotas were used in order to obtain adequate numbers of respondents for analysis 
by region and target groups (visible minority groups, Aboriginal people, persons with 
disabilities, and rural youth). 

The Youth Service Canada Phase II study was a follow-up study of a cohort of youth 
who applied to undertake placements between the summer of 1995 and March 1997. 
The sampling frame used in the Phase I study for both the participant and comparison 
groups was provided by a baseline survey conducted at the time of application to the 
program by project sponsors (see HRDC, 1999). Hence, in this study as in the Phase II 
follow-up study, non-participants were applicants who did not participate in YSC 
projects. Of the 1,242 youth who completed the Phase I survey (908 participants and 
334 non-participants), 636 were interviewed in Phase II (461 participants and 
175 non-participants). 

Survey statistics for the six evaluation studies are presented in Table 4-1. Fow response 
and small sample sizes in some of these studies should be kept in mind when considering 
the results. 
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Table 4-1 

Survey Statistics for Evaluation Studies of YES Components 
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58,600° 


41 0 e 


11,200° 


334 ' 


Invalid Numbers / 














No answer / 
Ineligible 13 


3,937 


3,937 


N/A 


138 


8,280 


133 


Refusals 


629 


629 


N/A 


44 


868 


26 


Completed 

interviews 


1,000 


1,000 


554 


228 


2,052 


175 


Program start 
period 


04/98-03/00 


04/98-03/00 


04/97-03/98 


04/97-03/98 


04/98-06/00 


06/95-03/97 


Survey period 


06/01-09/01 


07/01-09/01 


05/99-09/99 


03/01 


05/01-06/01 


01/00-02/00 



Participants according to annual reports of departments and agencies. 



b Examples of ineligible cases among participants are individuals who were on the list of participants but said 
they did not participate in the internship program and those who participated in another program between the 
end of the internship and the survey. Examples of ineligible cases among non-participants are individuals 
who eventually participated in an internship program or other major intervention, or were deemed not to be 
sufficiently similar to the profile of participants. 

c Number of potential comparison group members who had contact with an HRCC at the same period of time 
when participants entered the program according to administrative data. 

d Because of insufficient information on YIC projects involving 2 to 4 participants, only individual projects and 
projects involving 5 participants or more were included. 

e Respondents in the YIC Phase I survey who agreed to be contacted for a foilow-up survey. 

f Respondents in the YSC Phase I survey. 

N/A: not available. 

Source: Technical reports. 



4.2 Participant Profiles and Pre-program Characteristics 

This section presents the basic profiles of those participants in the YES components who 
responded to the surveys. The descriptive statistics are provided regardless of their 
statistical significance — they are statements regarding the composition of the samples 
and are not to be considered as estimates of the participant populations . 9 



9 In this section, groupings may overlap (e.g. a person may be a female and a member of a visible minority). 
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4.2. 1 Science and Technology 

The average age of S&T participants who responded to the survey was 25. Three-quarters 
had a post-secondary degree or diploma, and 60% had recently attended school prior to 
starting their internship. The most common area of study was science and technology. 

Female participants represented 44% of the survey respondents. Persons from visible 
minority groups accounted for 9% of the sample. Aboriginal people for 3%, and persons 
with disabilities for 2% (less than 10 respondents out of the total sample of 418). Almost 
two-thirds were never married, and four-fifths had no children living with them. 
All geographic regions were well represented. 

The respondents reported that in the year preceding the internships, they had spent one- 
quarter of their time, on average, in full-time employment, and slightly over one-third of 
their time in school full-time. The median earned income for participants in 1997 was 
$7,500, and in 1998, $10,000. In the 12 months preceding their internship, 20% of 
respondents had received El benefits, and 3% had received social assistance. Of those 
reporting some employment, 49% considered themselves underemployed. 

Slightly more than half (52%) of the respondents entered the program when they applied 
for a job and were then offered an internship. Approximately 25% had already worked 
for the employer in the past. Participants who were referred by an HRCC accounted for 
4%, while participants who were referred by another organisation or approached by an 
employer/sponsor represented 8% each. 



4.2.2 Youth International 

The average age of respondents who participated in YI was 25. Over 80% had a 
post-secondary degree or diploma, and more than half had recently (6 months or less) 
attended school prior to their internship. 

Female participants represented 58% of the sample. Persons from visible minority 
groups accounted for 15% of the survey respondents, Aboriginal people for 3%, and 1% 
were persons with disabilities (less than 10 respondents out of the total sample of 755). 
Almost four-fifths were never married, and only 7% had children living with them. 
All Canadian regions were well-represented. 

Respondents reported that in the year preceding the internships, they had spent, on 
average, just over a third of their time in full-time employment, and one-third of their 
time in school. The median earned income for participants in 1997 was $9,000 and 
$10,000 in 1998. Slightly more that 20% of respondents had received El benefits in the past 
12 months, and 3% had received social assistance. Of those who reported some employment 
before participating in the program, 63% considered themselves underemployed. 
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Approximately 42% of respondents entered the program by applying to a sponsor 
organisation after having heard of the internship. Approximately 22% entered the 
program after having been offered an internship when they applied for a job. 
Approximately 7% had already worked for the employer in the past. Participants who 
were referred by an HRCC accounted for 4%, while participants who were referred by 
another organisation or approached by an employer/sponsor represented 14% and 8% of 
the sample, respectively. 



4.2.3 Youth Internship Canada 

At the time of the Phase I survey, 10 the average age of respondents was 24. Most of 
them (52%) had a post-secondary degree or diploma, and 40% had last attended school 
less than 6 months before starting their internship. 

Female respondents represented 56% of the total sample of 1,439. Persons from visible 
minority groups accounted for 15% of the respondents, Aboriginal people for 9%, 
and 3% were persons with disabilities. Almost three-quarters were never married, 
and 22% had children living with them. The West was over-represented in the sample, 
while Quebec and the Atlantic region were under-represented. 

On average, respondents reported that in the year preceding their internships, they had 
spent 27% of their time in full-time employment, and 28% of their time in school. 
Of those reporting some employment before their internship, 62% considered 
themselves underemployed. 

The Phase III study examined a new cohort. The average age of the 2,05 1 respondents 
in this survey was 23. The proportion of respondents who had a post- secondary degree or 
diploma was 38%. About 41% had attended school less than 6 months before starting 
their internship. 

Female respondents represented 45% of the sample. Persons from visible minority 
groups accounted for 14% of the respondents, Aboriginal people for 11%, and 5% were 
persons with disabilities. Just over two-thirds were never married, and 23% had children 
living with them. Ontario and the West were slightly over-represented in the sample, 
while Quebec was adequately represented and the Atlantic region was under-represented. 

On the average, respondents reported that in the year preceding the internships, they had 
spent, 28% of their time in full-time employment and 2% of their time in school full-time 
(and an additional 3.5% of their time, on average, both in school and working part-time). 
In the 12 months preceding their participation, 15% of respondents had received El 
benefits, and 14% had received social assistance. Of those reporting some employment 
before the program, 53% considered themselves underemployed. 



10 The Phase II study analysed the same cohort surveyed in Phase I (although attrition caused differences between the 
characteristics of the two groups). 



Youth Employment Strategy (YES) 1997-2002: Summary of Recent Summative Evaluation Results 




Participants who entered the program after having applied for a job and, subsequently, 
being offered an internship represented 32% of the respondents in the Phase III survey. 
Approximately 16% had already worked for the employer in the past. Participants who 
were approached by an employer/sponsor accounted for 20%, while participants who 
were referred by an HRCC and those referred by another organisation represented 13% 
and 12% of the sample, respectively. 



4.2.4 Youth Service Canada 

At the time of the follow-up survey (on average, approximately 3 years after program 
completion), 51% of the 462 respondents who had participated in the program at the 
earlier period (between 1995 and 1997) were 23 years of age or over, and 49% were 
under 23. Slightly over 30% had a post- secondary degree or diploma when they entered 
the program. 

Female respondents represented 62% of the sample. Members of visible minority groups 
accounted for 19% of the respondents, persons with disabilities for 3%, and 2% were 
Aboriginal people (i.e., less than 10 respondents). Approximately 22% had children 
living with them. The sample was distributed proportionately among regions, although 
Ontario was slightly over-represented. 

Data on participants’ labour market activities in the pre-program period were not available. 



4.3 Summary of Results 

This section provides a summary of the main findings from the six evaluation studies on 
YI, S&T, YIC and YSC. These studies provide information on participant satisfaction 
with the programs; participant perceptions of the effects of participation in the programs, 
and estimates of the impacts of the programs as estimated through econometric 
techniques with respect to employment, unemployment, earnings, time in school, and use 
of EL 



4.3. 1 Participant Satisfaction with the Programs 

For the four programs, participant satisfaction was examined in some form. The S&T, YI 
and YIC Phase III studies asked survey respondents to rate their level of satisfaction with 
respect to several key elements of the programs. Respondents to the YIC Phase I survey 
were asked about their satisfaction with the level of pay during the internship, while 
respondents to the first YSC survey (see HRDC, 1999) were asked to rate their level of 
satisfaction with their internship in general. 11 The survey results are presented in 
Table 4-2, along with the completion rates (i.e. the percentage of participants who 
completed each program). 



1 1 The YIC Phase II study, a follow-up of Phase I participants, did not address these aspects of participant satisfaction. 
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A majority of program participants in S&T, YI and YIC (Phase III) who responded to the 
surveys stated that they were satisfied with the information they had received about the 
program and the application process. Between 73% and 89% of respondents were 
satisfied with these elements of the program. 

A majority of participants in these three programs were also satisfied with the job 
matching process, the opportunity to learn, and the length of the internship. The 
percentage of respondents who stated that they were satisfied with these elements of the 
program ranged between 83% and 94%. A majority of respondents (between 69% and 
78%) were satisfied with the rate of pay they received during their internship. 

For YSC, 92% of the participants who responded to the survey said that they were 
“fairly” or “very” satisfied with their internship. 

Program completion rates ranged from 79% to 88%. 

As discussed in the technical reports of the individual studies, according to other sources 
of information (focus groups and interviews), sponsors, employers and partners were also 
satisfied with these elements of the program. 



Table 4-2 

Participant Satisfaction* and Completion Rate (Based on Survey Respondents) 


Satisfaction (%) 
with 


Science & 
Technology 

(n=418)** 


Youth 

International 

(n= 755) 


Youth Internship Canada 


Youth 

Service 

Canada 

(n=908) 


Phase 1 

(n=1 ,439) 


Phase III 

(n=2,051) 


Internship in 
general 


N/A 


N/A 


N/A 


N/A 


92 


Information 
received about 
the program 


73 


83 


N/A 


84 


N/A 


Application 

process 


79 


89 


N/A 


87 


N/A 


Opportunity to 
learn 


94 


91 


N/A 


91 


N/A 


Length of 
internship 


86 


90 


N/A 


85 


N/A 


Matching process 


91 


86 


N/A 


83 


N/A 


Rate of pay during 
internship 


78 


69 


78 


74 


N/A 


Completion rate 
(%) 


87 


88 


82 


79 


87 


* In the S&T, YI and YIC Phase III surveys, satisfaction was measured on a 5-point ordinal scale; the table 
reports the percentage of respondents in who said they were “somewhat” to “very” satisfied (3 to 5 on the 
scale). A 7-point scale was used in the YSC Phase 1 survey and the table reports the percentage of 
respondents who said they were “fairly” or “very” satisfied (6 and 7 on the scale). 

** n is the sample size for the survey; the number of respondents to each question may be lower due to missing 
values. All percentages are rounded to nearest integer. N/A: not available. 

Source: HRDC (1 999) for data on YSC, technical reports for the other programs. 
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4.3.2 Participant Perceptions and Selected Outcomes 

All six of the studies asked program participants who also participated in the surveys 
about their perceptions regarding post-program outcomes (e.g. education, skill 
acquisition, and employment). Survey results are presented in Table 4-3. 

Three of the studies, the S&T, the YI, and the YIC Phase III studies, asked participants 
who responded to the surveys about their perceptions of the effects of the program on 
their decisions regarding return to school and career choices; and on their general level of 
employability, skills development, and match between their skill level and current job. 

Respondents to the S&T, YI and YIC Phase III surveys were asked about their general 
level of employability. They were asked to rate their employability prior to participating 
in the program, as well as their current level of employability at the time of the 
summative evaluation survey. Between 81% and 85% of them rated their current level 
of employability (their post program level at the time of the survey) as “high” or 
“very high”. When asked about their general level of employability at the time that they 
entered the program, only between 23% and 31% rated it as “high” or “very high”. 

Survey respondents in four of the studies were asked whether or not they felt that they 
enhanced or developed specific work-related skills during their involvement with the 
program. Generally, respondents indicated that they had. With respect to adaptability, 
over 90% stated that they had improved this skill. About 90% of respondents felt that 
they were more self-motivated than before the program, and between 86% and 93% felt 
that their teamwork skills had improved. 

For many, computer and numerical skills also improved. According to survey 
respondents’ self-assessments, the proportions who perceived it that way varied among 
programs. A much higher percentage (90%) of S&T participants stated that their 
computer skills had improved during their internship program, than participants (between 
55% and 67%) in other programs. A higher percentage of respondents to the S&T and 
YIC Phase III surveys indicated that their numerical skills had improved during their 
program (65% and 58%, respectively) than those in YI (41%). This appears to be in line 
with the objectives of the programs and the needs of the clients. 

Between 46% and 80% of S&T, YI and YIC survey respondents felt that their skills in 
mathematics and science and their technical research skills had improved during their 
participation in the internship program. 

Between 30% and 49% of respondents to the S&T, YI, and YIC Phase I and Phase III 
surveys stated that they returned to school after their internship. Among these 
respondents, 49% to 64% stated that the internship influenced this decision. 

A majority (between 78% and 85%) of S&T, YI, and YIC Phase III survey respondents 
indicated that the internship helped “somewhat” to “a lot” to determine their career choice. 

Respondents were also asked whether they considered themselves underemployed in 
terms of existing and most recent job for two periods: at program entry and at the time 
of the post-program survey. According to their self-assessments, the percentage of 
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respondents who stated that they were underemployed dropped over time. 
The percentage of respondents who stated that they were underemployed prior to or at 
entry to the program ranged from 49% to 68%; the percentage reporting underemployment 
at the time of the post-program survey ranged from 24% to 30%. 

A majority of respondents generally indicated that the internships were useful for getting 
their most recent job (between 72% and 86%). However, a majority of them (between 
65% and 77%) also believed that they would have found a job even if they had not 
participated in the programs. 

The proportion of participants who said they were unemployed and looking for work at 
the time of the post-program survey was consistent across components. It ranged 
between 10% and 13%. 

By design, the YES components provided participants with direct contacts with employers 
or sponsors. In many cases (between 48% and 56% of respondents), these contacts 
involved some form of mentorship and outlasted the internships. A majority of participants 
continued to have contacts with their program employers after the end of their internships 
(between 52% to 68%). However, as indicated earlier, many stated that they had already 
had contact with their internship employer prior to entering the program (the highest was 
52%, in the case of S&T). Moreover, certain participants had already worked for the 
employer prior to entering the program (the highest proportion was 25%, for S&T). 
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Table 4-3 

Participants’ Perceptions on Selected Outcome Measures (Based on Survey Respondents) 
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Table 4-3 (continued) 

Participants’ Perceptions on Selected Outcome Measures (Based on Survey Respondents) 
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4.3.3 Program Impacts on Employment Outcomes: 
Results of Econometric Analyses 

As stated earlier in Section 4.1, each of the studies included analysis of data collected in 
the surveys and the use of econometric techniques to address the question of program 
impacts. The econometric techniques were applied to the survey data in order to compare 
specific post-program outcomes of participants with those of non-participants. Table 4-4 
provides a summary of the differences that were estimated using the econometric 
techniques, by program and key employment-related variables: earnings, percent of time 
in full-time employment, percent of time in part-time employment, percent of time in 
unemployment, percent of time in school, and number of weeks on Employment 
Insurance. Similarly, Table 4-5 provides a summary of the estimated differences in 
earnings for women and men. 

The tables contain the point estimate and the 95% confidence interval associated with 
each (as discussed in Section 4.1.3 above). The abbreviation “N/S” that appears in some 
cells means that the study did not detect a difference between the two groups at the 95% 
level of confidence. The abbreviation “N/A” in other cells signifies that the study did not 
provide an estimate for that variable. 

The econometric analyses suggest the following: 

• As shown in Table 4-4, overall, earnings were estimated to be higher for program 
participants than they were for non-participants, except for participants in YSC. It was 
estimated that annualized earnings differences ranged from $2,299 to $5,724, 
depending on the program. The YSC follow-up study (approximately 3 years after 

program completion) did not detect a difference in earnings between the two groups at 

12 

the 95% level of confidence. 

• Table 4-5 provides the estimated differences between participants and non-participants, 
for males and females separately. The S&T study suggests that female participants gained 
$9,420 on average. The YI and YIC studies suggest that female participants gained $4,854 
and $3,400 respectively. According to the estimates, the S&T, YI and YIC studies did not 
detect earnings gains for men. The YSC study did not detect a difference in earnings for 
women and for men. 



12 Based on a survey conducted approximately 13 months after program completion, the YSC Phase I study 
(HRDC, 1999) found that participants had spent more time in school or training and less time employed or in 
the labour force than non-participants. These differences were no longer significant for the period between the 
Phase I and Phase II surveys. 
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• Regarding the percent of time employed, the studies provide mixed results. 
Participants in YI and YIC are estimated to have spent more time in full-time 
employment than non-participants (8.5% and 15.9% respectively). According to the 
estimates, there was no detectable difference between participants in S&T and their 
non-participant counterparts. Similarly, there was no detectable difference for the YSC 
participants and non-participants. While the analysis suggests that S&T participants did 
not spend more time in full-time employment than their non-participant counterparts, 
S&T participants did appear to spend more time in school than non-participants (5.7% 
more time per year, on average, on the part of participants). 

• According to the estimates, with the exception of YIC, the evaluations did not detect 
any difference in the number of weeks on El between program participants and 
non-participants (at the 95% level of confidence). 
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Table 4-4 

Estimated Difference* Between Participants in YES Components and Non-participants 
On Selected Post-program Outcome Measures (Based on Survey Respondents) 
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Table 4-5 

Estimated Difference* Between Participants in YES Components and Non-participants 
Earnings, by Gender (Based on Survey Respondents) 
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5. Overall Assessment and Conclusion 



The evaluations’ results are mixed. Generally, a majority of participants who responded to 
the surveys expressed a high level of satisfaction with key program elements. In addition, a 
majority of participants stated that participation in the program increased their 
employability and improved skills. Participants were also less likely to state that they were 
underemployed. However, many respondents did state that they would have found jobs 
even without the program, and many indicated that they had already had contact with their 
internship employer prior to the program. The econometric analysis on earnings, 
employment and unemployment suggests that programs had moderate employment-related 
gains for some youth. These tended to be the youth who participated in the streams that 
involved the more highly educated youth - the ones who participated in the Science and 
Technology, Youth International, and Youth Internship Canada streams. There were little 
to no gains for the group that participated in YSC before it was redesigned with greater 
focus on youth at risk. Overall, these results could support a reconsideration of selected 
aspects of the programs. 
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Appendix A: 
Evaluation Issues and Questions 



Issue 1- Relevance 

To what extent has the situation of youth evolved since the inception of YES, 
and does the program component address youth needs? 


Question 1 : To what extent are the early assumptions underlying the component still relevant? 


Question 2: What is the added value of having the component articulated as part of a stream under an 

interdepartmental, national ‘Strategy’ rather than a range of conventional separate departmental 
initiatives? 


Question 3: To what extent does the component address youth employment-related needs? 


Question 4: To what extent does the component meet the needs of employers? 


Issue II - Program Success 

To what extent is the stream effective in achieving its objectives? 


Question 5: To what extent did the program component assist youth in preparing for, obtaining and 

maintaining employment, and in making a successful transition into the labour market? 


Question 6: To what extent did participation in the program component result in increased employment, 

(including self-employment), improved earnings, and returns to school? 


Question 7: To what extent and for what reasons do some participants remain unemployed after participating 

in the program? 


Question 8: To what extent has the program established contacts between employers and youth? 


Question 9: To what extent has the program delivered measures to specific client groups (i.e. in terms of 

educational level, equity group status, rural/urban, regions, etc) and what were the impacts for 
each client group? 


Issue III - Program Cost-effectiveness 

To what extent is the stream the most appropriate and the most efficient 
means of achieving the stated objectives? 


Question 10: Are the activities under the program incremental, that is, over and above activities that employers 
would have undertaken in the absence of program funding? 


Question 1 1 : What are the costs and benefits of the program to society, governments and participants? 


Question 12: Is the program component an effective means of achieving program objectives and to what extent 
are the component's activities and characteristics factors in their success? 
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